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Static and Motion-Based Visual Features Used by Airport Tower Controllers: 
Some Implications for the Design of Remote or Virtual Towers 

Stephen R. Ellis and Dorion B. Liston 


SUMMARY 

Visual motion and other visual cues are used by tower controllers to provide important 
support for their control tasks at and near airports. These cues are particularly important for 
anticipated separation. Some of them, which we call visual features, have been identified from 
structured interviews and discussions with 24 active air traffic controllers or supervisors. The 
visual information that these features provide has been analyzed with respect to possible ways 
it could be presented at a remote tower that does not allow a direct view of the airport. Two 
types of remote towers are possible. One could be based on a plan-view, map-like computer- 
generated display of the airport and its immediate surroundings. An alternative would present 
a composite perspective view of the airport and its surroundings, possibly provided by an 
array of radially mounted cameras positioned at the airport in lieu of a tower. An initial more 
detailed analyses of one of the specific landing cues identified by the controllers, landing 
deceleration, is provided as a basis for evaluating how controllers might detect and use it. 
Understanding other such cues will help identify the information that may be degraded or lost 
in a remote or virtual tower not located at the airport. Suggestions are made regarding how 
some of the lost visual information may be displayed. Many of the cues considered involve 
visual motion, though some important static cues are also discussed. 


Introduction 

The visual cues necessary to fly and land an aircraft have been well studied over many decades (e.g. 
Gibson et ah, 1955; Grunwald & Kohn, 1994). In particular, the degradation in piloting 
performance and the consequent need to reduce airport capacity due to bad weather is fairly well 
understood. (FAA 71010.65R, 2006). The present report outlines a complementary side of the 
airport capacity-safety trade-off. It identifies and quantifies some of the visual features and 
properties used by tower controllers to monitor and enable safe landing and maneuvering on or near 
airports. These features are especially interesting due to recent proposals for technology and 
procedures in which controllers work in towers without a direct view of their controlled space. Such 
towers are described alternatively as a remote or “virtual tower” (JPDO, 2007). Work in these 
towers would be supported by controller displays of information about aircraft and the airport 
environment. 

In general, two types of displays can be considered: One would present a plan- view, map-like 
computer- generated display of the airport and its immediate surroundings (JPDO, 2007) similar to 
existing ASDE-x displays (Figure 1). An alternative would present a composite perspective view, 
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possibly provided by an array of radially-mounted cameras positioned at the airport in lieu of a 
tower (Fiirstenau, Mohlenbrink, Rudolph, Schmidt, & Halle, 2008), as shown in Figure 2. In either 
case, procedures and display techniques need to be developed which are cognizant of the current 
visual information used by controllers that may be lost. 



Figure 1. ASDE-x airport map display. 

The following discussion initially points out visual elements of the control task facing the tower 
evident in previous task analyses of tower operations (Paul, Zografos, and Hesselink, 2000; Werther, 
2006). However, this earlier work appears to provide only very general descriptions of the specific 
visual features to which controllers attend. To the extent the visual functions that are important to 
the controllers are considered, they are generally limited to questions of detection, recognition, and 
identification. The following discussion will examine other visual features that go beyond these 
basic three elements and relate in specific ways to the individual decision processes tower 
controllers develop to do their job; in particular, we discuss the motion of the controlled aircraft. The 
preliminary conclusion is that tower controllers use visual features to provide predictive position 
information allowing them to use anticipated separation to effectively and safely merge and space 
aircraft, and to maximize airport capacity. 



Figure 2. Out-the-window camera or synthetic vision display format. 
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Visual cues used by controllers are important for several reasons. First there is FAA interest in 
increasing airport capacity so that current operations under non-visual flight rules with reduced 
capacity may be modified to allow higher visual flight rules capacity during non-visual operations. 
For this purpose the currently used visual information needs to be provided by alternative means. 
Such “Equivalent Visual Operations” described FAA/NASA planning documents may be achieved 
with synthetic visual systems, i.e., (Kramer, Williams, Wilz, & Arthur, 2008) with replacement of 
direct tower camera or sensor views with visualized electronic position data. This replacement of the 
direct view, however, will not be fully successful and may be tragically misleading if the useful 
visual affordances provided by the real scene are not appropriately included or accounted for. 
Although Equivalent Visual Operations have primarily been considered from the pilot’s viewpoint in 
terms of flight displays that use new sensor data for synthetic vision, it has a flip side for which 
synthetic vision or camera-based displays could be used to present useful visual information within a 
remote or virtual tower. 

Significantly, this information need not be provided in the form of an image but could be provided in 
a more map-like plan view format and conceivably could even come along non- visual sensory 
channels, e.g. auditory or haptic. In fact, it could be based on data directly down-linked to ground 
displays from an aircraft indicating its state, i.e. spoilers deployed (Hannon, et al., 2008). 

The visual environment in an airport tower may be illustrated by considering the view from a 
specific tower such as that of San Francisco International Airport (SFO) (Figure 3). Such tower 
views show significant perspective compression at the ~1 nm range to runways and taxiway s, 
making commercial aircraft subtend small visual angles and posing viewing difficulties due to 
background visual clutter. Interestingly, during low visibility CAT III operations at SFO, airport 
operations may be conducted with the controllers never actually seeing the aircraft. Thus, since it is 
already possible for the controllers to continue many of their control tasks without visual contact, 
albeit with fewer aircraft, the idea of a remote tower may have some prima facie feasibility. But 
without visual contact, controllers must inform the pilot and those monitoring their communications 
that visual contact has been lost. Significantly, at the SFO tower where the parallel runways are 
~750 feet apart, continued operation without visual contact is associated with a loss (~50%) of 
airport capacity 1 . In contrast, at an airport such as Arlanda, Sweden (ARN) with the parallel runways 
~1 km (~3280 feet) apart, total loss of visual contact can have virtually no impact on capacity when 
the ground radar is fully functional 2 . Thus, there exist some operational examples of tower operation 
with total loss of visual contact. During low visibility operations it is not always necessary for the 
controller to maintain visual contact with the aircraft but for the aircraft to have enough forward 
visibility to safely maneuver the aircraft during ground taxi operations. 


1 Personal communication, ATCO, San Francisco International Airport, 7/7/2006. 

2 Personal communication, Tower Supervisor, Arlanda International Airport, 4/23/2007. 
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SFO Control Tower Partial Panorama 



SBA Control Tower Partial Panorama 



Stockholm-Arlanda Control Tower Partial Panorama 



Figure 3. The range of visibility of the airport tower’s immediate environments from unlimited 

visibility (San Francisco International, top) through partial occlusion due to low clouds 
( Santa Barbara Municipal, middle), to complete white-out (Stockholm-Arlanda, bottom). 


SFO Operations 

An analysis of the role of visual features in tower control can be developed from a more detailed 
discussion of operations for a particular airport, San Francisco International Airport (SFO). A sense 
of the overall strategy for some aspects of usual airport operation at SFO is best obtained from plan- 
view maps (See Figure 6 for SFO map). Aircraft are taxied from their gates to the southwest ends of 
runways 1L and 1R and launched in staggered pairs to the northeast. Departing aircraft are 
interleaved between aircraft landing on Runways 28 Left and 28 Right which also are treated as 
staggered pairs. Current winds, weather, and special operational requirements, of course, can 
significantly alter this pattern. For example, sometimes the longer 28 runways are needed for heavy, 
departing transpacific aircraft. Detailed descriptions of the alternative approach and departure 
procedures can be found in the Standard Instrument Departures (SID) and Standard Terminal Arrival 
Routes (STARS) associated with the airport but the local controller’s responsibility for arriving 
traffic generally begins with radio contact before the aircraft crosses the San Mateo Bridge and ends 
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for departing aircraft 1 nm beyond the end of the departure runway. By FAA rules, the local 
controller is generally responsible for aircraft entering and leaving the runways whereas the ground 
controllers handle, in a coordinated way, most of the taxiing to and from the gate. These two 
positions, in addition to that of the supervisor, are the ones that make the most use of the out-the- 
window information. The other two tower controller positions, Flight Data and Clearance Delivery, 
primarily use inside-the-tower information sources and voice communications. 

Visual Information Used in the Airport Tower 

The primary responsibility of the control tower is to ensure sufficient runway separation between 
landing and departing aircraft (FAA, 2006). A back propagating process may be used to understand 
the visual requirements supporting the tower controller’s primary responsibility. This process first 
identifies the visual affordances that the controller’s tasks involve. Affordances are the higher- level 
behavioral capacities that vision must support (Figure 4). Controllers, for example, must be able to 
identify the aircraft type, company, and flight status. They must control and recognize aircraft speed, 
direction, and position. They must establish a movement plan involving a succession of spatial 
goals. They must communicate this plan to the aircraft, coordinate it with other controllers and pilots 
as necessary, establish whether aircraft comply appropriately, and recognize and resolve spatial and 
other conflicts that may arise. These higher-level elements are supported visually by a number of 
visual functions: detection, recognition, and perception of the static and dynamic state of the aircraft. 
These functions are supported by still lower-level visual mechanisms: underlie luminance, color, 
control, position, and movement processing. These three levels of analysis provide a basis for 
describing the controller’s visual task. 


Transformations of Visual Information 

lower 


higher 
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Figure 4. Description of the dependency of the high-level spatial information 
needed by controllers on progressively low and lower perceptual 
functions and visual mechanisms. 
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The tower controller’s overall task has, of course, been analyzed within and outside of the FAA. It 
may be broken down into six different job subtasks: separation, coordination, control judgment, 
methods/procedures, equipment, and communication. Five of these subtasks involving vision have 
been identified by boldface type in Table 1 (Ruffner et al., 2003; FAA, 2006). 


Table 1. Analysis of Tower Control Tasks* 


Job Task 

Job Subtask 

1. Separation 

1. Separation is ensured and maintained at all times. 

2. Safety alerts are provided. 

2. Coordination 

1. Performs handoffs/point-outs. 

2. Required co-ordinations are performed. 

3. Control judgment 

1. Good control judgment is applied. 

2. Priority of duties is understood. 

3. Positive control is provided. 

4. Effective traffic flow is maintained. 

4. Methods/procedures 

1. Aircraft identity is maintained. 

2. Strip posting is complete/correct. 

3. Clearance delivery is complete/correct and timely. 

4. Letters of Agreement (LOAs)/directives are adhered to. 

5. Additional services are provided. 

6. Rapidly recovers from equipment failures and 
emergencies. 

7. Scans entire control environment. 

8. Effective working speed is maintained. 

5. Equipment 

1 . Equipment status information is maintained. 

2. Equipment capabilities are utilized/understood. 

6. Communication 

1 . Functions effectively as a radar/tower team member. 

2. Communication is clear and concise. 

3. Erses prescribed phraseology. 

4. Makes only necessary transmissions. 

5. Uses appropriate communications method. 

6. Relief briefings are complete and accurate. 


* Tasks inherently involving visual information are printed in bold. 


The assurance and maintenance of spatial separation is, of course, a visual task regardless whether 
separation is determined by radar or direct view. Handoffs and point-outs are also intrinsically 
dependent upon vision, though the need for the controller to adopt the pilot’s spatial frame of 
reference to direct attention toward objects and aircraft is also a significant cognitive task. Control 
judgment, being essentially a mental and cognitive issue, does not have an intrinsically visual 
component. But its connection with maintenance of effective and efficient traffic flow does 
emphasize the critical importance of time in traffic control. Three general methods and procedures 
directly involve vision: (1) establishment and maintenance of aircraft identify; (2) posting and 
correct annotation of flight strips; and (3) continual scanning of the entire control environment. 
Associated with these methods is the admonition to work quickly and to rapidly recover from errors 
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or off nominal conditions. Because each tower environment is to some extent unique, the specifics 
of procedures differ from tower to tower. All control techniques are, of course, consistent with the 
regulations cited and described in the FAA air traffic control (Order 7110.65R) but unique 
procedures and heuristics are passed on to future controllers by onsite training. The specific visual 
features tower controllers use can frequently be found in these locally developed heuristic rules. 

The overall tower control process has been formally analyzed and modeled including visual and 
nonvisual components (Alexander et al., 1989, Werther, 2006). For example, the MANTEA notation 
(Zografos & Hesselink, 2000) has been applied to analyze controller activity in the tower. Some of 
the elements identified in the MANTEA analyses are, in fact, visual but the visual components are 
only described in very general terms such as “visualize runway,” “visualize meteo,” etc. These 
descriptions only identify the sensory modality used to gather the information and are a general 
description of the content of the visual information but they say nothing specific about the actual 
visual viewing conditions or about the specific visual stimuli. This feature is common in other more 
recent and more sophisticated task analyses of visual features seen from the tower. Even the recent 
modeling done with Petri nets (Werther, 2006) does not identify specific visual stimuli but is more 
concerned with estimates of time required for the precision with which various visual sub-functions 
may be executed and to the logical conditions and consequences associated with the functions. 

The FAA has done some analysis of the specific visual performance expected from tower 
controllers. The work primarily focuses on the controller’s surveillance function and has been based 
on visual performance models developed for the military by CERDEC at Ft. Belvoir (e.g., 
Vollmerhausen & Jacobs, 2004). These models primarily are intended to predict the probability of 
visual detection, recognition, and identification of known targets. Detection refers to users’ ability to 
notice the presence of a particular object. Recognition refers to their ability to categorize the object 
into a general class such as a tank, light aircraft, or truck. Identification refers to their ability to 
determine the specific type of object, i.e., an Abrams tank, a Cessna 172, or a Ford refueling tanker. 
More modem similar visual performance models do not require the same amount of calibration 
techniques to determine model parameters for specific visual targets and specific users (Watson, 
Ramirez, & Salud, 2009). 

The CERDEC analysis, which predicts specific object perception from towers of various heights 
during a variety of atmospheric conditions and object distances, has been incorporated into a web 
tool to help tower designers ensure that specific architectural and site selection decisions for new 
towers will meet FAA requirements (Figure 5). Significantly, this tool focuses only on the 
surveillance function and does not address the aspects of visual motion that tower controllers use for 
the information, separation, and safety tasks. 
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Figure 5. The web interface to the FAA ’s tower design analysis tool that may be used by 

municipalities and others to test tower designs ultimately intended for FAA analysis and 
approval. Note website indicated in the upper right. 


In order to understand the details of the visual features used in tower control it is first necessary to 
identify the range within which controllers use visual information. We can use the example of SFO. 
Informal voluntary discussions and structured interviews with ten active controllers and supervisors 
who work at this tower were analyzed for the physical locations identified as points where various 
types of visual references are used while controlling approaching or departing aircraft. These 
discussions, which were considered preliminary work, were conducted with the knowledge and 
approval of the SFO tower manager, his chain of command, and the local NATCA representative. 
All primary notes were taken without personally identifying markings and transcribed into 
secondary statistical summaries or grouped data so as to preserve the anonymity of the respondents. 
Primary notes were thereafter discarded. 

These reported points where useful visual information could be seen primarily to include positions 
where visual contact with the aircraft is first or last were considered to be helpful. These positions, 
marked in Figure 7, include those for which aircraft come under or leave tower control, where they 
pass important ground references, or where visual contact provides other useful information. The 
points were determined independently from each of the controllers in response to the question 
“When you are in the Local controller position, where are the aircraft when you usefully observe 
them visually, what visual aspects of the aircraft do you observe, and why?” Controllers could 
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designate more then one point of interest for departing and more than one for arriving traffic; only 
two controllers took this option. One point represents nine controllers’ overlapping responses 
identifying approximately the same location about 1 nm beyond the end of the departure Runway 1. 
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Figure 6. SFO airport diagram showing typical 
movement paths for United Airlines, departures 
(dark/red paths) and arrivals (light green/paths) 



Figure 7. The first and last positions where 
SFO controllers report useful visual 
information with regards to landing ( Runway 
28) and departing aircraft ( Runway 1 ). The 
arrows show idealized, most common 
approach paths ( transparent green) to the 
west and departure paths (transparent red) 
to the north. 


In general, it is apparent from the distribution of points that controllers’ visual attention is much 
more spatially distributed to the aircraft approaching the 28LR runways and rather abruptly drops off 
about 1 mile off the end of the usual departure runways 1LR. These observations refer to the most 
common aircraft flow at SFO but suggest the generalization that the local controllers’ visual 
attention to approaching aircraft is distributed over a much larger area than that corresponding to 
departing aircraft. A likely reason for this is that departing traffic is handed off to approach/ 
departure control at 1 nm beyond the end of the runway and generally not thereafter of concern to 
the tower. 
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A significant aspect of the controllers’ remarks concerning when they first start paying visual 
attention, or when they last pay attention, to aircraft is that they rarely mentioned the aircraft’s visual 
motion 3 . One reason is that for the viewing angles and distances to the aircraft approaching and 
departing SFO, this motion is very small in terms of degrees per second. Often the azimuth rate is on 
the order of much less than 0.25°/second and rarely more than 0.57second. The visual accelerations 
are even much smaller and difficult to see because of atmospheric haze, thermal effects, and the 
visual range being beyond 5 miles. Visual rates of motion are more important for closer aircraft on 
or just seconds away from being on the runways or taxiways. 

Probably the most obvious need for visual contact by controllers in the tower is to immediately note 
unusual events that are not detected by electronic sensors such as radar. Examples could be heavy 
bird activity or an aircraft leaking fuel onto a taxiway. But there are a wide variety of other visual 
features that controllers use on a more regular basis when aircraft are close enough for the visual 
motion to be more easily noticed. Discussions with controllers have provided a list of some that are 
used (Tables 2 and 4). 

A tabulation (Table 2) of the visual features mentioned in the discussions with each of the SFO 
controllers shows the relative frequencies with which different features were mentioned. These 
discussions used a “cognitive walk-through” technique in which the controllers were asked to 
imagine representative approaching, departing, and taxiing aircraft under a variety of visual 
conditions and to report what they looked for visually to assist their control tasks. The consequent 
discussions were guided by the elements outlined in the Appendix. The most frequently mentioned 
features were relative motion between landing or departing aircraft and obstacles that could be on the 
runway. The first of these features is probably prominent because SFO has intersecting runways 
commonly used for takeoffs and landings. An assessment of all of the features mentioned, however, 
shows what may be a more general element. Seven of the 13 features identified in the interviews note 
that the feature helps the controller anticipate future activity. This information provides insight into 
pilot intent, knowledge, and likelihood of aberrant behavior. These predictive cues help the controller 
with the short term trajectory planning needed for anticipated separation and helps them allocate 
their attention to pilots either unfamiliar with the aiiport or maneuvering in unexpected ways. 


1 Visual motion is defined as the angular rate of change of the line of sight angle to an aircraft from the tower. 
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Table 2. Visual Features Identified by Interviews with 10 SFO Tower Controllers* 


Feature 

As 

Mentioned 

Commentary 

1 . Relative visual motion used 
to verify interleaving or 
takeoffs and landings. 

5 

Controllers verify their predicted separation of 
coordinated landing and approaching A/C by 
monitoring relative motion with respect to some 
stationary, visible direction of an object such as an 
airport light. 

2. Visual check obstacles or 
A/C for runway clearance. 

5 

Obstacle checks include ground vehicles, aircraft, 
birds, and people. 

3. Taxiing with ‘authority’ 
helps attention allocation. 

4 

Fast and ‘confident appearing’ A/C motion allows 
controllers to distribute their attention to pilots who 
appear unfamiliar land hesitant to maneuver so as to 
anticipate problems they may create. 

4. Aircraft attitude/altitude 
predicts ‘Go Around’. 

4 

Controllers like to be able to anticipate ‘go arounds’ 
by observing A/C attitude and altitude as various 
approach ‘gates.’ 

5. Visual speed, acceleration, 
or turn used to anticipate 
taxiway selection. 

4 

Controllers mentally integrate speed and acceleration 
(including turn rates) to anticipate future taxiways that 
might be used to complete A/C’s movement to for from 
gate. 

6. Coordinate/crosscheck 
visual and radar. 

4 

A large amount of time is spent crosschecking visual 
separation during approach and departure with radar 
information during VFR conditions. 

7. Visible wing dip predicts 
turn. 

3 

Visible banking given a quick prediction/confirmation 
that A/C is turning in conformance with clearance. 

8. ‘Mike and a mile’ rule for 
interleaving takeoffs and 
landings. 

3 

Predictive rule: A/C needs to be rolling across taxiway 
Mike on RW 1 with matched landing A/C on RW28 at 1 
nm final for the required separation to be obtained. 

9. Engine smoke and heat 
confirms takeoff roll start 

2 

Modern A/C don’t smoke much and have cooler exhaust. 

10. Onset of navigation lights 
or strobe predicts coming 
dynamic change. 

2 

Appearance of these lights allows controller to 
anticipate call from a/C requesting clearances and 
instructions. 

1 1. Visual resolution of 
motion and position better 
at airport than radar. 

1 

Near the tower (<l-2 nm) the visual display of the real 
world has many more ‘pixels’ than associated radar 
displays. 

12. Visual check done on tail 
for A/C company. 

1 


13. Check landing gear. 

1 

This check is done so automatically by controllers that 
it wasn’t mentioned due to focusing of the interview on 
visual features for separation. 


* Boldface marks out the predictive aspect of specific visual features 
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Visual Features at SFO and Other Airports 

In order to examine the generality of the visual features and produce a list as complete as possible, 
structured anonymous interviews were conducted with controllers from an additional seven airports. 
Because we were not able to obtain timely agreement from the national NATCA office for the 
participation of line controllers, these additional discussions were limited to supervisory personnel. 
Anonymity was maintained since all written notes were taken without personally identifying 
markings and formal questionnaires were not used. To ensure anonymity, original notes were 
transcribed into statistical or grouped secondary notes and the originals were thereafter discarded, 
ensuring that no personally identifiable information was recorded or could be reconstructed post hoc. 
In all cases, tower visits to U.S. airports were conducted with the knowledge and approval of the 
specific tower’s manager and FAA headquarters. In addition to that of San Francisco International 
Aiiport (SFO), U.S. airport towers that were visited were: Boston International (BOS) MA; Golden 
Triangle Regional (GTR) MS; Santa Barbara Municipal (SBA) Santa Barbara, CA; and Norman Y. 
Mineta San Jose International (SJC), San Jose, CA. Supervisory controllers from Denver 
International (DEN) Denver, CO, LaGuardia Airport (LGA), New York City, NY, and Philadelphia 
International (PHL) Philadelphia, PA were included in the multi-airport analysis. They visited the 
first author at NASA Ames Research Center and provided information regarding the nature and 
location of visual features used by controllers while viewing airport diagrams and regional maps. 

The tower at Stockholm- Arlanda (ARN) in Sweden was the only foreign airport tower visited but 
was not included in any quantitative analysis. Table 3 gives a summary of the airport towers 
considered and the personnel interviewed. 


Table 3. Airport Tower Environments Discussed and Evaluated 


Airport Tower Environments Discussed 

Number of 
Controllers or 
Supervisors 

Notes 

Stockholm (Arlanda) ARN 

1 

Discussions were held but visual 
features from the ARN tower were not 
analyzed. 

Boston International (BOS) 

3 

Supervisors only. 

Denver International (DEN) 

1 

Supervisor only without airport view. 

Golden Triangle Regional (GTR) 

1 

Supervisor only. 

La Guardia International (LGA) 

1 

Supervisor without airport view. 

Philadelphia International (PHL) 

1 

Supervisor without airport view. 

Santa Barbara City (SBA) 

2 

Supervisors only. 

San Jose International (SJC) 

3 

Supervisors only. 

San Francisco International (SFO) 

11 

One supervisor, 10 controllers. 

Total 

24 
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Figures 8 and 9 illustrate how the visual velocity of aircraft viewed from the tower could be 
determined for moving aircraft at or near the airport and those that were farther away in the airport 
vicinity but still visible. Figure 10 provides a breakdown of various classes of features as 14 general 
categories that were used to organize the features. Counts on the numbers in each category give an 
idea of their relative frequency of mention. At this stage of investigation no systematic attempt was 
made to determine the relative operational importance or frequency of use of the various features. 
Investigations are currently underway in collaboration with Jerry Crutchfield of the Civil Aerospace 
Medical Institute (CAMI) to determine the frequency of use and criticality of the visual features that 
have been identified 4 . (Also see van Schaik, Roessingh, Lindqvist and Fait, 2010.) In particular, the 
high frequency of mention of the points of first and last useful visual contact are undoubtedly an 
artifact of their mention in the structured interview as an example of the kind of visual information 
being sought. The point of the investigation was to collect as broad a range of visual features as 
possible for further analysis in subsequent studies that are presently underway. 
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Figure 8. Lines of sight from the San 
Francisco International Airport (SFO) 
tower to positions on the airport 
where the visual motion was analyzed. 
Simple geometry allows calculation of 
rates of change of lines of sight from 
the tower to aircraft from knowledge 
of tower and aircraft position and 
aircraft velocity. 


Figure 9. Lines of sight from the Boston 
Logan International Airport (BOS) tower 
to positions in the airport region where the 
visual motion of moving aircraft were 
analyzed. 


4 The project is called Concurrent Validation of AT-SAT for Tower Controller Hiring (CoVATCH). AT-SAT stands for 
Air Traffic Selection and Training test battery. 
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Inventory of Visual Velocity of Features used for 
Traffic Control at Eight* Commercial Airports II 



Figure 10. Inventory of visual velocity for features used for traffic control. 


When a controller identified a visual feature, its location was plotted on an appropriate map. 
Afterwards, the direction of flight and speed was determined from the appropriate airborne traffic 
pattern or ground path. Simple geometric analysis was then possible to determine the apparent visual 
rate of the aircraft as seen from the tower at the time the visual feature would have been noted. 
Because actual aircraft speed was not actually measured, speed was estimated from typical rates 
mandated by approach procedures or estimated by controllers and pilots familiar with the airport and 
typical air and ground aircraft motion. Some reflection on the geometry shows, however, the aircraft 
speed to have a comparatively small influence on visual motion. Its impact is dwarfed by the effect 
of relative direction of flight. An aircraft flying directly towards the tower can have virtually 
0°/second visual velocity! The relative direction of flight used for analyses was determined from the 
interviewees and the typical patterns of motion at and around the airport if the original notes did not 
include the needed information. Once the approximate visual velocity associated with each visual 
feature was determined, a spectrum of visual velocities associated with each of the 14 feature 
categories could be determined. These are shown in Figure 1 1 and summed to give an overall total. 
These spectrums of visual velocity for each of the categories of features reflect some of the physical 
aspects of each category. The first and last useful visual contact rates are slowest because these are 
in general the farthest from the tower. Visual rates during landing deceleration are high because the 
aircraft are generally closer to the tower yet still moving relatively fast compared to taxiing. 

For the purposes of the present inventory the most important aspect of the distribution of motions is 
not its shape or arithmetic mean but its mode and range. As can be seen in Figure 1 1 , the vast 
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majority of visual rates are less than 1 “/second with the mode at a small fraction of a degree/second. 
These visual rates are quite slow compared to those typically studied in visual psychophysics. If a 
concept of operations for a remote or virtual tower is to include visually presented targets that 
provide the information that controllers currently pick up from aircraft motion, then the display 
techniques need to be able to represent this range of slow motion for visual cues that controllers 
currently use. It is important to note that the useful presentation of aircraft motion therefore benefits 
significantly from the use of very large format displays. To the extent that the display scales down 
visual motion due to screen size, the displayed visual rates, which are already very slow, could well 
become imperceptible and require special signal processing to be operationally useful. An example 
of such processing could be the computational detection of the slow motion and its denotation by 
introduction of or changes in visible symbology. A second important caveat is that the visual rates 
are not seen in isolation but have a temporal context; in fact, the change in visual velocity itself can 
be an important cue which is identified for some visual features in Table 4 and discussed in more 
detail in the final section. 



0 0.5 1 1.5 2 2.5 3 3.5 

Visual Velocity (deg/s) 


Figure 11. Visual velocities associated with each of the feature categories. 


Table 4 provides a summary of all the visual features identified from discussions with controllers 
from all analyzed airports. It lists the identified visual feature, the information the feature provides 
the controller, and suggests some general information support characteristics that would be 
necessary to provide equivalent information on alternative displays that might be used in a virtual or 
remote tower: (1) a map-like display that could be driven by ground radar or other comparable 
positions information, e.g. ADSB; and (2) an image-like display that resembles the out-the-window 
view from a tower and could be driven by airport cameras or other sensors and computer graphics 
providing synthetic vision (Figures 1 and 2). 
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Table 4. Visual and other Perceptual Features that Aid Tower Air Traffic Control 

Visual Feature 

Visual Information 
Provided 

Corresponding Decision 
Support Information and 
Display Techniques for 
Map-like Displays 

Corresponding Decision 
Support Information and 
Display Techniques for 
Out-the -window Image- 
like Displays 5 

Status 

1. A/C is prepositioned with an 
anticipatory rotation for a turn 
while holding short of a 
taxiway or runway. 

Pilot is correctly expecting 
to be cleared for a specific 
turn. 

Current and static A/C 
orientation should be 
shown on electronic 
map. 

Visual resolution of 
display should be 
sufficient for user to 
recognize A/C pose at 
crossing points. 

2. A/C type. 

Predicts likely ground 
acceleration, e.g. the 
difference between turbine 
vs. constant speed propeller 
A/C determines separation 
techniques used. 

A/C type should be 
indicated by icon shape 
or data tag to relieve 
controller memory load. 

High resolution visual 
image required to support 
existing visual 
performance requirements 
for tower design. 

3. Dust up or thermal optical 
distortion from thrust. 

Applied power can confirm 
compliance with take-off or 
other clearances that require 
engine spool-up. 

Down-linked indications 
from A/C of engine 
spool up should be 
displayed on A/C icon. 

Evidence of spool up 
should be visible on 
display or A/C icon 
associated with the power 
up should be displayed 
based on down-linked 
information. 

4. Smoke, spray from wheel 
indicates ground contact and 
touchdown point. 

Touch down point, landing 
likely unless a touch-and-go 
is planned. Helps to identify 
likely taxiway to be used to 
exit runway. 

Down-linked 
information from wheel 
sensors indicating 
touchdown should be 
displayed on A/C icon to 
indicate touchdown 
point. 

Visual evidence of wheel 
contact should be visible 
or down-linked 
information from wheel 
sensors indicating 
touchdown should be 
displayed on A/C icon. 

5. Navigation lights being turned 
on. 

Call to tower is imminent, 
usually to the Clearance 
Delivery Controller at a big 
tower. 

Down-linked 
information regarding 
cockpit A/C start up (i.e. 
before engine start) 
should be displayed (e.g. 
A/C icon first appears on 
display on startup before 
pilot calls tower). 

Navigation lights when 
A/C is in the gate should 
be visible. Down-linked 
information regarding 
cockpit A/C start up (i.e. 
before engine start) should 
be displayed if visibility is 
insufficient, example, e.g. 
A/C icon first appears on 
display on startup before 
pilot calls tower. 


5 Synthetic vision or image type displays can in general also be augmented with computer generated icons or data tags in 
what would be called an augmented reality display. In contrast to an electronic map display, the choice to use an image 
type display could be based on a minimal sensor system using only cameras so as to as to keep costs and computational 
overhead low. 
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6. A/C relation between A/C 
attitude and altitude. 

The visual relationship 
between A/C attitude and 
altitude predictive of pilot 
intent such a landing or 
executing a missed 
approach. 

A/C pitch attitude should 
be displayed 
geometrically or 
numerically for 
comparison with speed 
display with short delay 
< ~1 sec. 

Pitch attitude and speed 
need to be perceivable on 
display with short delay < 
~1 sec. 

7. Reflected “lights” on the water. 
Visible reflections of A/C 
light off ground features such 
as bodies of water or a 
runway surface that confirm 
normal or indicate deviant 
flight path. 

At some airports reflections 
of landing lights off 
surfaces like water can 
independently confirm 
normal lateral position and 
orientation of landing A/C; 
such information is similar 
to pilot reports of passing 
the Outer Marker. 

Indication of A/C 
passing over “virtual” 
markers along approach 
route and outer or inner 
marker shown on 
display, possibly sourced 
from data down-link. 

Visual fidelity of image 
of approaching A/C 
should include large 
specular reflection of 
landing lights. 

8. A/C mechanical status, gear, 
flaps, spoilers, reversers. 

Confirms appropriate 
aerodynamic status of A/C. 
Confirms intention to land. 
Can be used to indicate 
onset and intensity of 
braking, predicting the A/C 
deceleration profile. 

Down-linked data from 
A/C should provide data 
for display of status of 
gear, flaps, spoilers, and 
reversers to confirm 
commitment to landing. 

Aerodynamic 
configuration of A/C 
should be visually evident 
or enhanced by graphic 
overlays based on down- 
linked data. 

9. Weather conditions 

immediately at airport (e.g. 
fog, rain, water on runway). 

Cross check pilot reports, 
provide weather 
information, determine 
VFR/IMC status, determine 
airport approach/departure 
patterns, provide input for 
ATIS. 

Map symbology should 
include weather icons 
and/or text indications 
based on down-linked 
A/C or airport sensor 
information. 

Weather should be 
visually apparent on 
display or presented by 
overlaid icons and text 
based on down-linked or 
airport sensor 
information. 

10. Weather conditions near 
airport (e.g. ceiling, RVR, 
Outer and Middle Markers). 

Cross check pilot reports, 
provide weather 
information, determine 
VFR/IMC status, determine 
airport approach/departure 
patterns, provide input for 
ATIS. 

Map symbology should 
include weather icons 
and/or text indications 
based on down linked or 
airport sensor 
information. 

Weather should be 
visually apparent on 
display or presented by 
overlaid icons and text 
based on down-linked or 
airport sensor 
information. 

11. First/last visual acquisition. 
The position where an 
approaching aircraft is 
normally first usefully visible 
or where visibility is typically 
lost for a receding aircraft. 

Confirm location of radar 
contact, spacing w/r to A/C 
in pattern. 

Display A/C icon 
corresponding to initial 
and final radar contact. 

Provide sufficient visual 
contrast and resolution to 
allow visual contact at 
times and positions 
comparable to view from 
a real tower. 

12. Movement during taxi. 

Verify compliance with taxi 
clearance and/or detect 
violation. 

A/C motion and position 
need to be observable. 
Note: Because of 
reduced display size and 
map scale, the physical 
motion on the display 
may be below perceptual 
thresholds. 

A/C motion and position 
need to be observable. 
Note: Because of reduced 
display size, the physical 
motion on the display 
may be below to 
perceptual thresholds. 
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13. Animal obstructions or 
intrusions. 

Need to issue obstruction 
warning, modify approach, 
departure, or ground 
movement. Could be as 
small as a snapping turtle or 
as large as a bear. 

Airport sensor data (e.g. 
motion sensors or 
cameras) should be used 
to provide timely 
displays of obstruction 
locations and 
movements. 

Visual displays should 
have sufficient resolution 
and contrast to match out- 
the-window views. 
Airport sensor data (e.g. 
motion sensors or 
cameras) could 
alternatively be used to 
provide timely iconic or 
text overlays. 

14. Birds, flocks, large birds. 

Need to issue bird activity 
warning, modify approach, 
departure, ground 
movement. 

Airport sensor data (e.g. 
motion sensors or 
cameras) should be used 
to provide timely iconic 
and/or text displays of 
obstruction locations and 
movements. 

Visual displays should 
have sufficient resolution 
and contrast to match out- 
the-window views. 
Airport sensor data (e.g. 
motion sensors or 
cameras) could 
alternatively be used to 
augment display to 
provide timely iconic or 
text warning overlays. 

15. Inanimate obstacles on 
runway/taxiway. 

Need to issue obstruction 
warning, modify approach, 
departure, ground 
movement, possible 
communication with user- 
operated vehicles. 

Airport sensor data (e.g. 
motion sensors or 
cameras) should be used 
to provide timely iconic 
and/or text displays of 
obstructions locations 
and movements. 

Airport sensor data (e.g. 
motion sensors or 
cameras) should be used 
to provide timely iconic 
and/or text displays of 
obstacles or displays 
making them visually 
detectable. 

16.Unexpected/unanticipated 

event. 

Visual observation of event 
requiring nonstandard/ 
emergency procedures. 

Not handled well 
without sensors designed 
for unanticipated 
dangers; consequently 
rare but dangerous 
events could be missed. 

High visual fidelity wide- 
field-of-view surveillance 
with high sample rate and 
low latency required for 
unanticipated events, 
which likely have a visual 
component. 

Acceleration/Deceleration 

17. A/C beginning visual 
acceleration of takeoff 
roll. 

Confirms compliance with 
Clearance to takeoff. 

Detection of onset of takeoff roll 
by low latency motion sensors. 
Down-link from A/C or other 
sensors would be needed to 
provide information with delays 
comparable to current view of 
the A/C. Note: Physical size of 
map display will make initial 
A/C motion harder to see than 
direct out-the-window view (see 
text). A discrete onset of motion 
signal on the map, such as 
making the A/C symbol double- 
bright, would greatly assist 
controllers. 

High resolution, 
bandwidth, low latency 
view of A/C starting 
takeoff roll is required 
for visual confirmation 
of compliance. Such a 
display could provide 
information for 
equivalent to the current 
out-the-window view. 
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18. A/C landing deceleration 
reportedly sensed to 
anticipate use of high- 
speed turnoff. 

Predicts length of landing 
roll and, indirectly, the 
turnoff and taxiways used 
after exiting the runway. 

Down-link from A/C or other 
sensors would be needed to 
provide information with 
delays comparable to current 
view of the A/C. Note: 
Physical size of map display 
will make initial A/C motion 
harder to see than direct out- 
the-window view. 
Consequently, a ground 
speed data tag should be 
associated with the landing 
A/C. It could be removed at 
the end of the landing roll. 

High resolution, 
bandwidth, low latency 
view of A/C starting 
takeoff roll is required 
for visual confirmation 
of compliance. Such a 
display could provide 
information for 
equivalent to the current 
out-the-window view 
allowing controllers to 
use current perceptual 
speed estimation 
techniques. Large visual 
displays would need to 
be used to present rates 
of visual angles 
comparable to current 
visual contact. 

19. A/C pitching after main 
gear touch down. 

Predicts use of aerodynamic 
braking, length of landing 
roll, and indirectly the 
taxiway to be used to exit 
runway. 

Down-link from A/C or other 
sensors would be needed to 
provide information with 
delays comparable to current 
view of the A/C. A visual 
indication on the landing A/C 
icon of nose wheel contact 
could provide comparable 
information. 

High resolution, 
bandwidth, low latency 
view of A/C landing A/C 
is required for visual 
confirmation of pitch 
down. Large visual 
displays would need to 
be used to present rates 
of visual angles 
comparable to current 
visual contact. Current 
specifications for tower 
design provide adequate 
visual requirements for 
the visibility of A/C 
pitch that could be 
adapted for remote/ 
virtual towers. 

20. A/C pitching after landing 
braking. 

Predicts landing, length of 
landing roll, taxiway to be 
used to exit runway and 
related to assigned gate. 

Down-link from A/C or other 
sensors would be needed to 
provide information with 
delays comparable to current 
view of the A/C. A visual 
indication on the landing A/C 
icon of nose wheel contact 
could provide comparable 
information. 

High resolution, 
bandwidth, low latency 
view of A/C landing roll 
is required for visual 
detection of pitching. 
Since this pitch cue is 
smaller than that at touch 
down its visibility on 
out-the window displays 
should be verified. 
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2 1 . A/C pitching during 
initiation of takeoff 
(especially B757). 

Confirms compliance with 
Clearance to Takeoff. 

This information is redundant 
with the indication of onset 
of takeoff roll (see above). 

High resolution, 
bandwidth, low latency 
view of A/C starting 
takeoff roll is required 
for visual detection of 
pitching. Since this pitch 
cue is smaller than that 
at touch down, its 
visibility on out-the 
window displays should 
be verified. 

22. Banked wing predicts turn 
faster than change in A/C 
position. 

Confirms compliance with 
Clearance. 

Aircraft symbol or data tag 
needs to indicate A/C pose. 

High resolution, 
bandwidth, low latency 
view of A/C banking is 
required for visual 
detection of pose. 

23. A/C initiating turn onto 
taxiway, especially cue 
from nose wheel angle. 

Confirms clearance to turn 
onto taxiway, nose wheel 
angle predicts turn. 

Down-link from A/C or other 
sensors would be needed to 
provide information with 
delays comparable to current 
view of the A/C. A visual 
indication on the landing A/C 
icon of nose wheel angle and 
A/C pose w/r to taxiway and 
runway could provide 
comparable information. 

High resolution, 
bandwidth, low latency 
view of A/C taxiing is 
required for visual 
detection of pose and 
nose wheel position. 

24. Timing of visible plume 
effects of thrust reversers 
and spoilers, Note: These 
cues are distinct from the 
visibility of the 
mechanical deployment of 
these devices. 

Predicts landing 
deceleration, length of 
landing roll, taxiway to be 
used to exit runway and 
related to assigned gate. 

Down-link from A/C or other 
sensors would be needed to 
provide information with 
delays comparable to current 
view of the A/C. A visual 
indication on the landing A/C 
icon of deployment of thrust 
reversers could provide 
comparable information. 

High resolution, 
bandwidth, low latency 
view of A/C landing roll 
is required for visual 
detection of deployment 
of reversers and spoilers 
(see text). 
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Speed 

25. Visual deviation of glide 
path seen as relative 
motion against stationary 
reference. Relative motion 
of an A/C seen against 
stationary ground 
references, allowing its 
glide path to be more 
easily perceived. 

Confirms correct approach/ 
departure paths. 

Graphical display of flight 
path against a ground- 
referenced map could provide 
some comparable visual 
information but the 3-D 
element would require an 
AGL-based altitude data tag 
for the A/C icon. 

High-resolution visual 
image required based on 
existing visual 
performance 
requirements for tower 
design. 

26. Relative motion of 
visually overlapping 
targets. Relative motion of 
visually, partially 
overlapping objects that 
allows them to be 
perceptually separated 
(e.g. two aircraft along 
approximately the same 
line of sight). This cue is 
especially helpful at night 
when A/C are seen as 
light patterns. 

Breaks visual clutter, aids 
perceptual separation of 
otherwise confusing objects. 

Relative motion can also be 
displayed on a map but the 
sampling rate degrades and 
delays motion perception. 
De-clutter algorithms can be 
employed to remove clutter. 
The usual plan-view format 
minimizes clutter due to 
perspective compression seen 
from a tower. 

High resolution, 
bandwidth, low latency 
view of visually 
overlapping A/C and 
background is required 
for visual judgment of 
relative motion. Current 
specifications for tower 
design provide adequate 
visual requirements for 
the perception of relative 
motion (see text). 

27. Relative motion of A/C on 
crossing trajectories with 
respect to a fixed ground 
reference such as a lamp 
pole. 

Confirms correct approach/ 
departure paths, allows 
estimation of safe passing 
through runway 
intersections such as those 
at SFO. 

Stationary ground reference 
symbols should be introduced 
to map displays to make the 
relative motion of moving 
symbols easier to perceive. 

High resolution, 
bandwidth, low latency 
view of visually 
overlapping A/C and 
reference objects is 
required for visual 
judgment of relative 
motion (see text). 

28. A/C speed during taxi, 
“Taxing with authority.” 

Speed indicates level of 
pilot familiarity with airport 
and likelihood of clearance 
conformance, improves 
distribution of controller’s 
attention, unusually slow 
speed indicates need for 
special attention. 

Ground speed data tags 
should be associated with 
A/C symbols. If such data 
tags are not provided, the 
physical map size needs to be 
large enough that high and 
low speed taxiing can be 
distinguished by controllers. 

High resolution, 
bandwidth, view of taxi 
area required for visual 
judgment of motion. The 
physical size of the 
display needs to be 
sufficient for 
discrimination of high 
and low visual rates of 
taxiing (see text). 
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Sound 6 

29. Sound of takeoff power. 

Confirms compliance with 
takeoff clearance. 

Directional sound cues 
provided by 360° radially- 
mounted directional 
microphones should be pro- 
vided within a remote tower. 

Directional sound cues 
provided by radially- 
mounted directional 
microphones should be 
provided within a remote 
tower. 

30. Sound of engine run-up. 

Preparing for takeoff. 

Directional sound cues 
provided by 360° radially- 
mounted directional 
microphones should be 
provided within a remote 
tower. 

Directional sound cues 
provided by radially- 
mounted directional 
microphones should be 
provided within an remote 
tower. 

31. Loud unexpected sound. 

Attention directed to source; 
possible explosion, bomb, 
attack etc.; important 
adjunct to visual 
information. 

Directional sound cues 
provided by 360° radially- 
mounted directional 
microphones should be pro- 
vided within an remote 
tower. 

Directional sound cues 
provided by radially- 
mounted directional 
microphones should be 
provided within a remote 
tower. 

Additional Observation 

32. General surveillance. 

Some airport towers are 
strategically placed so as to 
provide useful, excellent 
visual surveillance outside 
of the airport and relevant 
airspace. 


The field of regard may 
be usefully made larger 
than that needed for A/C 
control for airports where 
general surveillance is 
needed (e.g. Boston, 
Logan). 


6 In discussions of visual features used to aid control, many controllers spontaneously mentioned the importance of 
sound cues, so we have included them in this table. 
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A better understanding of exactly how some of these cues can be used can come from examining 
them quantitatively. In the next section an example of such analysis is presented with respect to 
landing deceleration at SFO. 

Deceleration during Landing at SFO 

In order to analyze the deceleration of aircraft landing at SFO, digital video images were recorded of 
the initial braking after touch down. Recordings of a wide variety of landing aircraft were made to 
examine a wide range of decelerations. The 45 observed and reported aircraft included 747-400s, a 
variety of models of 767, 757, 737, A3 19, A320, CRJs, and small twin turboprops. The weather was 
clear with light winds from the west. The landing data from all the aircraft have been aggregated 
since there was no intention to make a more detailed analysis by type but rather to understand the 
range of visual rates and visual decelerations that would be visible from the airport tower. 

The following analysis begins to determine the magnitude of this visually sensed deceleration and 
how it could be used by controllers. Through this process we identify one of the dynamic visual 
features used in traffic control from the airport tower: the change in speed evident during a single 
glance a controller might make towards a decelerating landing aircraft 7 . In thinking about what 
specific aspects of the visual stimulus to which the controllers might be attending, it is helpful to 
remember that perceptual discriminations of commonly experienced magnitudes of sensory 
quantities such as velocity are fairly well described by Weber’s Law, which states that the just 
noticeable difference (JND) is a constant proportion of the quantity’s magnitude. This so-called 
Weber fraction is roughly constant for a variety of psychophysical parameters but under the best 
conditions is ~6 % for changes in velocity viewed within a typical 0.5-second time period. For 
stimuli with random mixtures of spatial frequencies, i.e. mixtures of contours of different sizes, the 
JND grows to about 7.5%. Very significantly for the very slow visual velocities less than 1 
degree/second such as those commonly seen from the control tower for landing and departing 
aircraft, the JND can climb up to ~10% ( McKee, Silverman, & Nakayama, 1986). 

It is therefore important to understand that controllers may not be directly sensing the visual 
velocities per se even though they may claim to do so. They may, in fact, develop alternative 
viewing strategies allowing them to translate speed into displacement during relatively fixed time 
intervals, thus making the detection of unusual rates of change easier. Additionally, alternative 
visual cues to quantities such as deceleration could be used. For example, aircraft pitch while 
moving along the ground could be equally well a clue to the onset or offset of braking. 

It is not so much the visual aspect of the visual information that is important as it is the fact that the 
information revealed by vision is relevant, real, direct, unmediated, immediate, and continuous that 
makes it the best basis for the best possible anticipation of future action. This is why the visual input 
could be critical. Replacements for it need to capture the same predictive, informational features as 
suggested in Table 4. 


’During normal vision, people make from 3-5 fixations per second (Rayner & Castelhano, 2007). However, when 
studying some aspect of an ATC image, fixations duration can increase but rarely grow longer than approximately 1.3 s 
(e.g. Remington, Lee, Ravinder, Matessa, 2004). Consequently, a reasonable constraint for modeling the duration of a 
controller's glance would be to insure that they are 1.3 s or less. 
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In order to begin to analyze the visual features actually present in real landings in more detail, we 
have initially focused on the deceleration profile of aircraft landing on the 28 Left and 28 Right 
runways at SFO. Controllers report that they use their sense of degree and timing of this specific 
deceleration to anticipate which taxiway would be needed for the aircraft to exit the active runway. 
Their decision is time critical during heavy runway use since landing aircraft are staggered in pairs 
and interleaved with departures on crossing runways 1R/1L. 

We made 15 frame/second video recordings at 1024 x 768 resolution of the braking phase of 45 
aircraft landing on 28L and 28R and processed the recordings to measure changes in visual velocity. 
We used a custom MatLab® image processing technique that isolated the moving contours across a 
set of two frames and averaged them to localize the aircraft and provide their screen velocity in 
degrees per second. Using the viewing geometry described in Figure 12, we have recovered the 
aircraft braking profile and computed the changes in its visual velocity as viewed from the control 
tower by re-projecting the movement, as it would have been seen from the tower. Thirty of these 
velocity profiles (low pass filtered with a 1Hz cutoff) are shown in Figure 13. 




Figure 12. Camera parameters and view at SFO. Markers at known ground positions 
determined from Google™ Earth ground images were used in combination 
with the known geometry of the runway to convert line of sight angles to 
aircraft from the camera position into position along the runway and 
thereafter into line of sight angles from the airport tower and thereafter into 
visual velocities as seen by controllers. 
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Figure 13. Line of sight for A/C at SFO. 


Because of the noise present in our current recording technique, we were unable to obtain velocity 
and acceleration values with acceptable noise levels. We were, however, able to obtain a directly 
recorded braking deceleration profile 8 for another A3 19 aircraft landing on Runway 28L from the 
same company, comparably loaded and flying in the same wind and weather conditions as one of the 
aircraft we had recorded visually. Since we knew the touchdown points for these two A3 19 landings, 
we’ve combined the two trajectories to produce what we believe to be a fairly accurate landing 
profile as seen from the tower (Figure 14). 

The deceleration profile in Figure 14 shows the aircraft approaching and passing the tower as it 
decelerates. In fact, during the approach the visual velocity actually increases during the deceleration 
because of the decreasing distance between the aircraft and the tower. It is clear from the 
deceleration profile that there are several phases of braking due to deployment of the thrust 
reversers, spoilers, and mechanical brakes and further data collection and processing needs to be 
done to more precisely identify these periods. However, the very smooth velocity plot in Figure 14 
(third panel from top) already shows that the amounts of velocity change in the braking within any 
short time window 2 seconds or less are well less than the ~6% usual Weber fraction for a just 
noticeable difference of midrange psychophysical quantities such as perceived speed. This level is 
defined by convention to be that difference in a sensory quantity that can be detected correctly 75% 
of the time and is therefore not evidence of a very strong sensory stimulus 9 . This observation leads 
to some skepticism that the controllers are detecting velocity change per se because controllers 
would likely wish to be more certain regarding their judgments than 75% correct. Accordingly, they 
may have developed a strategy to detect speed change by some other means, perhaps by comparing 
displacement for approximately equal time periods. Such a timing strategy might be evident in eye 
tracking records of controllers judging aircraft deceleration. Of particular interest will be future 


8 The aircraft’s deceleration was recorded just after touch-down using a arm rest stabilized iPhone in Airplane Mode 

running an application called Motion Data with sampling rates at 30 Hz. 
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analyses and experiments to determine how well the controller’s sense of aircraft deceleration can be 
maintained with airport imagery spatially degraded by pixilation and sensor noise, and temporally 
degraded by low sampling rate. The sampling rate issue has been addressed by research currently 
being prepared for publication (Ellis, Fuerstenau, & Mittendorf, 2011). 
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Figure 14. Line of sight changes. 


Conclusions 

1. Aiiport tower controllers use visual features observed during aircraft operations to provide 
information beyond simple detection, identification, and recognition of aircraft. 

2. Twenty-eight useful visual features have been identified from discussions with 24 controllers and 
supervisors. Some involve the static pose of the aircraft of interest but many of the most useful 
involve aircraft motion, especially aircraft acceleration and deceleration. 

3. The visual features provide predictive or lead information regarding future aircraft position, pilot 
intention, and pilot airport familiarity that enable controllers to appropriately distribute their 
attention during operations and to anticipate possible conflicts. 

4. The very slow rates of visual motion in terms of subtended visual angle suggest that the change in 
velocity reported by controllers is not directly sensed but must be observed by learned viewing 
strategies developed from tower experience. 

5. Directional aircraft sounds audible in the tower are also used to assist operations. 
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Appendix: Structured Interview Form 

The following outline was used to structure the interviews with tower managers, supervisors, and 
controllers in order to elicit the visual features they observe in the course of tower operations. This 
structure was primarily intended to spark a conversation about visual information used during 
operations during different kinds of aircraft flows. Discussions were generally held in rooms with a 
clear view of ongoing tower operations. The cues were generally collected with respect to the 
Ground Controller and the Local Controller positions. 

Raw notes were taken during each individual discussion with the information transferred to 
notations made on airport and regional maps on which notes from multiple discussants were 
cumulated. The primary notes were then discarded to preserve controller anonymity. 

The sequence of the outline was usually used, but discussions were not restricted to following it. 

Discussion Topics/Activities for Structured Interviews 

1. Aiiport immediate surroundings 

a. Draw approach and departure paths into and out of airport for different flows/time of 
day/weather conditions. 

b. Note points of initial and final useful visual contact: comment on with respect to lighting, 
specific visual conditions. 

c. Imagine A/C flowing past visual references along the approach paths, identify important 
behaviors, controller rules of thumb useful for conformance monitoring. 

d. What information and or procedures would be lost if out-the- window vision is lost? 

2. Aiiport 

a. Draw approach and departure paths into and out of airport for different flows/time of 
day/weather conditions. 

b. Note points of initial and final useful visual contact: Comment on with respect to lighting, 
specific visual conditions and other useful visual reference points. 

c. Imagine A/C flowing past visual references along the approach paths, identify important 
behaviors, controller rules of thumb useful for conformance monitoring. 

d. What information and or procedures would be lost if out-the- window vision is lost? 
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